General Dynamic Programming Algorithmsapplied to Polling

We formulate the problem of scheduling a single server in a multi-class queue-ing system as a Markov decision process under the discounted cost and the average cost criteria. We develop a new implementation of the modiied policy iteration (MPI) dynamic programming algorithm to eeciently solve problems with large state spaces and small action spaces. This implementation has an enhanced policy evaluation (PE) step and an adaptive termination test. To numerically evaluate various solution approaches, we implemented value iteration and forms of modiied policy iteration, and we further developed and implemented aggregation-disaggregation based (replacement process decomposition and group-scaling) algorithms appropriate to controlled queueing system models. Tests provide evidence that MPI outperforms the other algorithms for both the discounted cost and the average cost optimal polling problems. In light of the complexity of implementation for the aggregation-disaggregation based algorithms and their performance, we cannot recommend them for problems with a sparse transition probability matrix structure. Our new implementation of MPI is eeective in discounted problems and better than previous implementations under the average cost per stage criterion.